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ABSTRACT 



A number of procedures were used to evaluate the 
effectiveness of the "Helping Hand, " a free monthly newsletter published by 
the Barbour County (Alabama) Extension Service, with a circulation of 1,000. 
Seven criteria were established using related literature and advice from 
extension service professionals. A questionnaire and a cover letter were 
developed and validated by a formative committee, field-tested for 
readability, and mailed to 100 of the newsletter's recipients. After 
telephone follow-up, 61 responses were collected. The results revealed that 
95 percent of the clientele read almost all of the newsletter, 61 percent 
would be disappointed if the newsletter failed to arrive, and 33 percent 
would notice its absence. Most respondents indicated that they always found 
the newsletter useful, reliable, and timely. The also indicated that the 
newsletter was easy to understand, and 80 percent would like to receive it 
monthly. The study concluded that the newsletter is an effective method of 
reaching individuals who do not have time to take courses, attend meetings, 
or read books. A follow-up survey was recommended. (The report contains 18 
references and 5 appendixes that include the criteria of the survey, the 
formative committee, the questionnaire, the cover letter, and a copy of the 
newsletter.) (KC) 
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Abstract of a practicum report presented to Nova Southeastern 
University in partial fulfillment of the requirements 
for the degree of Doctor of Education 



AN EVALUATION OF THE EFFECTIVENESS OF 
THE HELPING HAND NEWSLETTER IN 
BARBOUR COUNTY, ALABAMA 
By 

Habiba N. Shaw 
February, 1996 

The Barbour County Extension Service is a public-funded, 
ormal , education network that links Auburn University's 
knowledge base in agriculture, home economics, and community 
development to the people and communities of Barbour County. 

H. S Ipxng — Hand is a free monthly newsletter that reaches one 
thousand clientele in Barbour County, Alabama. The problem was 
that no determination of the readers 1 utilization of the 
newsletter had been done. The purpose of the study was to 
evaluate the effectiveness of the Helping Hand . The three 
research questions for the study addressed the appropriate and 
valid criteria, questions, and format for this survey. 

An evaluation methodology was used. Eight procedures were 
executed. Seven criteria were established by reviewing related 
literature and advice from three extension service professionals. 
A questionnaire and a cover letter were developed and validated 
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by a formative committee. The questionnaires and the cover 
letter were field-tested to assure readability. 

The mail-out package consisted of the questionnaire, a cover 
letter, a pre-addressed, postage-paid return envelope, and a copy 
of Hglpinq H^nd . The packages were mailed August 16, 1995. Due 
to a low response rate, follow-up telephone calls and home visits 
were made to encourage more responses. Data from sixty-one 
respondents were collected. The results revealed that 95% of the 
clientele read almost all of the newsletter. Sixty-one percent 
indicated that they would be disappointed if the newsletter 
failed to arrive, and 33% would notice its absence. Respondents 
were asked to rate the Helpin g Hand in terms of its usefulness, 
reliability, and timeliness of information on a Likert scale. A 
vast majority of the respondents indicated that they always found 
the newsletter useful, reliable, and timely. They also indicated 
that the newsletter was easy to understand, and 80% would like to 
receive the newsletter monthly. 

It was concluded that the newsletter is an effective method 
of reaching individuals who do not have time to take courses, 
attend meetings, or read books. 

Recommendations included that Barbour County Extension 
Service should continue publishing the Helping Hand and conduct a 
follow-up survey. 
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Chapter 1 
INTRODUCTION 

The Barbour County Extension Service (BCES) is a statewide 
continuing education network that links Auburn University's 
knowledge base in agriculture, home economics, and community 
development to the people of Barbour County. Extension 
education is a process of working with people to help them gain 
and apply knowledge, solve their problems, and improve the 
quality of their lives. Extension education programs are an 
off-campus, informal (noncredit) educational effort guided by 
specific objectives and including activities and events that are 
planned, conducted, and evaluated for their impact on 
participants' learning needs. The extension's educational 
activities include meetings, field days, workshops, 
consultations, media programs, presentations, discussions, 
correspondence courses, and newsletters. 

Helping Hand is a free monthly newsletter that reaches one 
thousand clientele in Barbour County, Alabama. The County 
Extension Home Economist has been using this newsletter to 
disseminate research-based nutrition information since 1983 . 

Nature of the Problem 

For several years preceding this study, federal funding for 
extension programs had been reduced. As a result, county 
extension offices were being asked to evaluate programs and set 
priorities for their clients' needs. Extension educators had to 
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deliver information more efficiently than ever before. 

Substantial time, effort, and a cost of $4,488.00 annually were 
expended by the extension staff in preparation of the newsletter. 
The problem was that no determination of the readers ' utilization 
of the newsletter has been done . 

Purpose of the Study 

The purpose of this study was to evaluate the effectiveness 
of the Helping Hand newsletter. Specifically, the objectives of 
this study were as follows: ' 

1. To ascertain readers' use of the Helping Hand : 

2. To ascertain readers' opinions of the format, 
appearance, and content of the Helping Hand : 

3. To identify recipients who no * longer wished to receive 
the Helping Hand . 

Significance to the Institution 

A major challenge confronting the extension system is that 
of determining the impact of its programming efforts in bringing 
about clients 1 behavioral change and a substantial improvement in 
the quality of life of its public. This evaluation assisted BCES 
in reaching decisions on the future direction, design, and 
funding of the newsletter and whether its publication should be 
terminated, altered, or maintained. 

Relationship to Seminar 

Human Resources Development (HRD) is concerned with learning 
strategies that continuously improve the performance of the 
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organization's human resources, both individually and in groups. 
The study of HRD involves learning about effective approaches to 
strategic human resource planning, alternative methods for 
improving performance, technological transfer of learning, 
evaluation, and the return on investment to the organization from 
such activities (Baskett, Carnes, Groff and Sample, 1994, p. 7). 
The evaluation process of this study followed these principles of 
the HRD seminar. 



There were three research questions for this study: 

1. What are the valid and appropriate criteria to evaluate 
the newsletter? 

2 . What are the valid and appropriate questions to measure 
the readers' attitudes, the usefulness of the newsletter, and the 
suitability of the newsletter contents? 

3 . What is the valid and appropriate format for the survey? 



The following terms are defined to add clarity to this 
s tudy . 

Checklist item . A checklist item is simply a method of 
providing the respondent a number of options from which to 
choose . 

Clientele . The clientele are the people who plan and 
participate in the extension's educational programs. 

Closed-ended questions . Closed-ended questions (also called 



Research Questions 



Definition of Terms 



ERIC 




10 



structured questions) are best for obtaining information and data 
that can be categorized easily (McMillan and Schumacher, 1993, p. 
243) . 

Descriptive research . Descriptive research describes 
systematically the facts and characteristics of a given 
population or area of interest, factually and accurately (Issac 
and Michael, 1995, p. 50) . 

Double-barrelled questions . Double-barrelled questions 
contain two or more ideas, and frequently the word "and" is used 
in the term. Double-barrelled questions and statements are 
undesirable because the respondent may, if given an opportunity, 
answer each part differently (McMillan and Schumacher, 1993, 
p. 240) . 

Likert scale . A true Likert scale is one in which the stem 
includes a value or direction and the respondent indicates 
agreement or disagreement with the statement (McMillian and 
Schumacher, 1993, p. 244). 

Newsletter . A newsletter is a publication that provides 
specific information directly to specific clientele, including 
information on the latest research developments, including 
results, innovations, and techniques that are relevant to the 




clientele . 
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Chapter 2 



REVIEW OF LITERATURE 



The purpose of this study was to evaluate the effectiveness 
of the Helping Hand newsletter. The literature review mainly 
examined the readership research that used a descriptive survey 
design in the form of a self-adininistered, mailed questionnaire. 
The factors relative to the evaluation were identified and 
reviewed. These factors included: (a) appropriate criteria for 

the evaluation, (b) appropriate questions to measure readership 
patterns and reasons for reading the newsletter, and (c) the 
establishment of appropriate format for the survey. 



In 1992, Suvedi and Johnson conducted a readership survey to 
get feedback on the Communicator . a monthly newsletter of the 
Michigan State University Extension Service. The objectives of 
the study were to ascertain readers' utilization of the 
publication and their perceptions about the format, appearance, 
and major stories in the newsletter. The study used a 
descriptive survey sent through the U.S. mail. The data 
collection instrument was a self -administered questionnaire. The 
results indicated that the majority of the respondents preferred 
to receive a monthly printed newsletter (pp. 1-20) . 

A readership survey of the Messenger . another newsletter 
from the Michigan Agricultural Experiment Station (MAES) , was 
conducted by Suvedi et al . (1993) . The primary objective of that 
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survey was to get responses from newsletter readers in order to 
improve and strengthen internal communication between MAES staff 
members on policies and programs. This study also used a 
descriptive survey in the form of a mailed questionnaire. 

Overall, respondents indicated that they preferred to receive 
information through newsletters (pp. 1-12) 

By using a simple, inexpensive, but carefully designed 
postcard survey, Smith and Kiernan (1992) evaluated 
The Woodlander . a monthly newsletter from the Cooperative 
Extension Service of Penn State University. A postcard 
containing four questions was sent to 10% of the 1500 
recipients to measure the effectiveness of the newsletter. 

Because a substantial majority of the Respondents indicated 
that they almost always read all of the newsletter, the agent 
decided to continue the newsletter in its current format and 
style (p. 31) . 

Young families in Pondera County, Montana, improved their 
resource management skills by the Getting it Together newsletter 
series. This newsletter was a pilot project; therefore, two sets 
of evaluation approaches were used. The first evaluation was 
based on the interaction sheet included with each newsletter. 

The second approach was a five-page questionnaire sent to all its 
participants . The evaluations revealed that respondents had 
improved their resource management skills through the newsletter 
and that they would prefer to receive similar newsletters from 
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the extension system (Goetting and Pourroy, 1991, pp . 17-19) . 

Ludwig (1988) reports a survey of a newsletter titled 

£ ihgi$- Parent News , developed by the Ohio Cooperative Extension 

Service . The newsletter had been used by agencies as a teaching 
tool, posted on bulletin boards, and requested by many single 
parents. Parents were asked to give their reactions to the 
newsletter in addition to the questionnaire data. The survey 
indicated that the newsletter was an effective tool for reaching 
single-parent families with needed information in a form very 
usable to them and at a convenient time (p. 23) . 

Extension agents in two Maryland counties evaluated the 
Triple E News; Eat, Exercise, Extend Your Life , a monthly 
newsletter for the dif f icult- to-reach senior citizens in isolated 
parts of the community. A descriptive survey was designed to 
test the feasibility of the newsletter. The data collection 
instrument was a self-administered questionnaire. 

The evaluation indicated that most of the seniors were 
interested in good nutrition and physical fitness. They shared 
newsletters with friends and neighbors and looked forward to 
recipes for small servings. When newsletters were late, agents 
would get eager calls about their newsletters (Frazier, Collins, 
and Rhodes 1991, pp. 34-35) . 

A team of six Cooperative Extension family life specialists 
conducted a national survey of the use of Acre-paced parent 
education newsletters by extension programs. Information was 
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collected by a questionnaire sent to the extension family life 
specialist responsible for parent education in each state. 
Specialists from 19 states reported use of the Aae-paced parent 
education newsletters. 

Three of the states in the survey were selected to furnish 
evaluation data from parents who had received their Acre-paced 
newsletters. In those states, over 95% said that the newsletter 
had helped them learn more about infant growth; over 65% said 
that it helped them relate to their baby; and 73% said that it 
improved their self-confidence as a parent. More than 50% said 
that it helped them care more effectively for their own needs 
(Nelson and Cudaback, 1985, pp . 13-14) . 

In an attempt to understand the factors that influence the 
use of computers to access agricultural information databases, 
Schumacher (1989) surveyed the subscribers of Doane 1 s 
Zk qrigultural Newsletter regarding their usage patterns. Cover 
letters and questionnaires were sent to all 5,000 subscribers. 

By collecting data from 391 respondents, the researcher was able 
to ascertain some of the difficulties of using "AgLine" and other 
information databases. As a result of the survey, the author 
recommends an educational focus to enhance future use of these 
agricultural information services (pp. 131-137). 

Piovane (1995) conducted a survey to evaluate the quality of 
the Kutztown University's alumni publication, The Tower , through 
a readership survey. The evaluation process was developed using 
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a modified version of Dillman's total design method (the process 
includes a personalized cover letter, an attractive, simple to 
complete survey, and at least two follow-up contacts) . The 
survey yielded a 32% response rate. Due to the low response, 
accurate generalizations about the larger alumni population and 
their opinions could not be made. The vast majority of the 
respondents perceived the Tower to be a quality publication (pp. 
57-66) . 

Pavlik, Vastyan, and Maher (1990) examined why employees 
reai i Vital — Signs, the employee newsletter at the Hershey Medical 
Center. The researchers selected a sample of 1,000, or roughly 
one- third of the total employee population. The questionnaire 
consisted of three main parts. First part measured readers' 
patterns on a five-point scale, the second part assessed the 
importance of different reasons for reading the newsletter, and 
the final section solicited background information on the 
characteristics of each employee. The survey revealed that 
overall readership of the newsletter was fairly high, with more 
than two- thirds (67.6%) reading most or every issue of Vital 
Signs. The study also revealed that pictures were the most 
regularly ''read" item in Vital Signs (pp. 53-55). 

Designing Survey Instruments 

In evaluating the effectiveness of publications, several 
factors can be addressed through a survey (Shoemaker, 1990) . 

These factors include (a) to what degree a publication is read, 
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(b) if it is read, how readers evaluate the quality of the 
publication, and (c) whether the readers find the articles 
related to their interest (pp. 42-44) . 

A readership survey can be utilized to assess the quality of 
wr i^ing in a publication, the value of the newsletter as an 
information source, and the identification of potential feature 
articles . 

According to Fisher (1993) the purpose of the readership 
survey is to obtain a "general idea of what the reader value" 

(p. 43) about the publication. Fisher believes that mail and 
telephone surveys are effective survey methods and these methods 
promote a higher response rate. 

Paxson (1992, p. 197) and Stratton and Angerosa (1995, pp. 
46-51) suggest that the survey questionnaire should be simple, 
clear, concise, and easy to complete. A respondent will be 
comfortable in completing the questionnaire and not consider the 
task to be time-consuming if the survey appearance is less 
threatening. In order to make the task appear brief, an 
attractive layout should be used, the number of questions should 
be minimized, and repetition be should eliminated. Paxson (1992) 
and Stratton & Angerosa (1995) also suggest that a stamped self- 
addressed return envelope be included with the letter and the 
survey. This helps the respondent in a number of ways. First, 
it eliminates a minor financial constraint. Second, it saves 
time and effort on the respondent's part to do this task. Third, 
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it can be perceived as a benefit or reward by the respondent. 
Pretesting the cover letter and the questionnaire with a small 
sample of respondents is also recommended. 

A carefully designed follow-up sequence is essential for a 
good response rate (Faria & Dickinson, 1992, pp. 57-60; Paxson 
1992, p. 197; Stratton & Angerosa, 1995, pp . 46-51) . Three timed 
contacts are recommended. Each follow-up differs somewhat from 
its predecessor since new and stronger appeals are made to 
persuade a respondent to complete and return the survey. 

Summary 

The review of literature has provided a framework for 
conducting a readership survey through a self-administered, 
ma ilsd questionnaire. The literature review identified several 
readership surveys conducted by the extension professionals in 
different states and by other researchers (Suvedi and Johnson, 
1992; Suvedi et al . , 1993; Smith and Kiernan, 1992; Goetting and 
Pourroy, 1991; Ludwig, 1988; Nelson and Cudaback, 1985; Frazier, 
Collins, and Rhodes, 1991; Schumacher, 1989; Piovane, 1995; 

Pavlik, Vastyan, Mahar, 1990) . 

The research review also includes few suggestions for 
designing survey questionnaires (Shoemaker, 1990, pp. 42-44; 
Fisher, 1993, p. 43; Paxson, 1992, p. 197; Stratton and Angerosa, 
1995, pp. 46-51; Faria and Dickinson, 1992, pp . 57-60). 
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Chapter 3 

METHODOLOGY AND PROCEDURES 
Methodology 

The study employed an evaluation methodology to solicit 
information from Helping Hand readers regarding their assessment 
of the publication. The purpose of the study was to evaluate the 
effectiveness of the newsletter. The objectives of this study 
were (a) to ascertain readers' use of the HglP^ n( 3 Hand/ (k) to 
ascertain readers' opinions of the format, appearance, and 
content of the Helping Hand , and (c) to identify recipients who 
no longer wish to receive the newsletter . 

Procedures 

Eight procedural steps were used to complete this evaluation 
practicum. 

First, a review of literature was conducted. The review 
included an overview of readership surveys and designing survey 
instruments . 

Second, criteria were established for the evaluation. 

The criteria were based on the information collected from the 
literature review and on advice from a county extension agent, 
an extension specialist, and a research associate from the 
extension service (see Appendix A) . 

Third, a data collection instrument was developed. The 
survey instrument was a two-page questionnaire. A jury of 
experts was appointed to review the questionnaire to ascertain 
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face and content validity. The expert panel consisted of a 
county agent from an adjoining county, a county agent 
coordinator, and a district agent from the state headquarters 
(see Appendix B) . 

Fourth, the questionnaire was field-tested among ten people 
to assure readability and uniform understanding of survey 
questions (see Appendix C) . 

Fifth, the population for this survey consisted of 10% of 
the 1,000 recipients on the newsletter's mailing list. Through a 
stratified random sampling process, 65 people were selected from 
three rural communities and 35 people were selected from one 
urban community. In this procedure, the entire population on the 
mailing list was divided into subgroups, on the basis of gender 
and age. After the population had been divided, samples were 
drawn randomly from each subgroup . 

A cover letter (see Appendix D) , questionnaire (see Appendix 
C) , a copy of the newsletter (see Appendix E) , and a postage-paid 
return envelope were mailed to one hundred clientele. The cover 
letter explained the importance of the survey, guaranteed 
confidentiality of responses, and appealed for a prompt response. 
The packet was mailed to the identified sample on August 16, 

1995. 

Sixth, as following measures, a telephone survey and home 
visits were conducted to secure a higher response rate. 

Seventh, data were analyzed by using descriptive 
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statistics (i.e., by frequency and percentage of response) . 

Eighth, a report including conclusions and recommendations 
was submitted to the administrator of the Tuskegee University 
Extension Service. 

Assumptions 

For this practicum, it was assumed that the panel of 
experts had the knowledge to validate the questionnaire for 
this survey. It was further assumed that data provided by the 
10% of the newsletter recipients would represent the opinions 
of the total readership. 

Limitations 

The study was limited to an evaluation of the Helping Hand 
newsletter in Barbour County, Alabama. The results would not 
necessarily apply in other settings. 
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Chapter 4 
RESULTS 

Three research questions were presented to evaluate the 
effectiveness of the Helping Hand newsletter. 

1. What are the valid and appropriate criteria to evaluate 
the newsletter? 

2. What are the valid and appropriate questions to measure 
the readers* attitudes, the usefulness of the newsletter, and the 
suitability of the newsletter contents? 

3 . What is the valid and appropriate format for the survey? 

The initial procedures used to develop the criteria included 

a review of related literature and advice from three extension 
service professionals. The second phase of the project included 
the development of the survey questionnaire and formation of a 
panel of experts to ascertain the face and content validity of 
the questionnaire. The third phase involved field-testing the 
questionnaire to assure the readability and uniform understanding 
of survey questions. The final phase of this project included 
mailing the questionnaire to 10% of the 1,000 recipients on the 
newsletter's mailing list and collecting data. 

Review of Related Literature 

The review of the related literature partially answered the 
first research question. The appropriate criteria to evaluate a 
newsletter were addressed by Paxson (1992, pp. 197), Stratton & 
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Angerosa (1995, pp. 46-60), Fisher (1990, pp . 42-44), and 
Shoemaker (1990, pp. 24-26) . 

The following references were reviewed to answer the second 
and third research questions. 

1. Several readership surveys were studied. Included in 
this area were readership survey instruments using closed-ended 
questions (Schumacher, 1989, pp . 131-137; Pavlik, Vastyan, and 
Mahar, 1990, pp. 53-55; Piovane, 1995, pp . 57-66) . 

2. A review of several extension newsletter evaluations 
were included in the literature review (Suvedi and Johnson, 1992, 
pp. 1-20; Suvedi et al, 1993, pp . 1-12; Smith and Kiernan, 1992, 
p. 31; Goetting and Pourroy, 1992, pp. 17-19; Ludwig, 1988, p.23; 
Frazier, Collins, and Rhodes, 1991, pp. 34-35; Nelson and 
Cudaback, 1985, pp . 13-14) . 

3. A meticulous literature review was also conducted to 
identify and to develop the format of the survey (Paxon, 1992, 
p.197; Stratton and Angerosa, 1995, pp. 46-51; Faria and 
Dickinson, 1992, 57-60). This review provided guidelines for the 
appropriate format of the survey. 

Criteria for the Survey 

To establish the criteria for the survey, a literature 
review was conducted on several readership surveys. To establish 
criteria, the researcher also sought advice from a county 
extension agent from Barbour County Extension Service, an 
extension specialist from Tuskegee University, and a research 
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associate from Tuskegee University (see Appendix A) . They were 
each provided with a summary of the literature review, a sample 
of a readership survey that was conducted at Michigan State 
University by Suvedi et al . (1993), and a copy of the handbook on 

"research and evaluation" by Isaac and Michael (1995). 

Selection of Goals 

After their review, discussion, and comments, the advisers 
first decided to establish the following five goals to guide the 
development of the survey questionnaire: 

1. Do clientele read the newsletter, Helping Hand ? 

2 . How important is the information in the Helping Hand 
to them? 

3 . How do clientele evaluate the quality of the newsletter? 

4. How do clientele evaluate the content and format of the 
newsletter? 

5. How many clientele wish to receive the newsletter? 

Selection of Criteria 

At the same meeting, the group decided to set the following 
eight criteria for developing the survey questionnaire: 

1. Each question should be related to the research goals. 

2. Questions should be relevant. 

3. Questions should be clear, concise, and unambiguous. 

4. Avoid double-barrelled questions. 

5. Respondents must be competent to answer. 
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6. Avoid negative items or terms. 

7. Avoid biased items or terms. 

Development of Data Collection Instrument 
The questionnaire was developed to meet the goals and the 
objectives of the study. To develop the questionnaire for the 
survey, a literature review and a review of several readership 
surveys were conducted. The questionnaire was designed according 
to the established criteria. 

Draft Survey Questionnaire 

The following six questions were developed initially for the 
survey : 

1. How much of an issue of the Helping Hand do you 
usually read? 

All or almost all 

At least half 

Less than half 

Little or none 

2. If your copy of the Helping Hand failed to arrive each 
month, would you: 

Miss it 

Notice its absence 

Be able to function without it 

Never know the difference 
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3 . Please rate the Helpi ng Hand in terms of the following 
statements : 

Always Sometimes Rarely Never 

Information given in the 

Helping Hand is useful __ 

Information given in the 

Helping Hand is reliable 

Information given in the 

Helping Hand is timely 



4. Please check your reaction to this statement: The 

newsletter is easy to understand; unnecessary and 
difficult words are avoided. 



Strongly agree 

Agree 

Not sure 

Disagree 

5. It costs about $4,488 per year to print 1,000 copies of 
the Helping Hand for 12 months. Which of the following 
statements comes closest to expressing the way you feel 
about this expenditure? 




26 



26 

— T he Helping Hand is an excellent way to receive 
information from extension service. The current cost is 
within reason. 

— The Helping Hand provides new and reliable 
information, but the cost is too high. 

The Helping Hand has outlived its usefulness and 

should be discontinued. 

6. Is a printed newsletter your preferred way to receive the 
information? Which of the following statements comes 
closest to your feeling? 

I prefer to receive a monthly newsletter. 

I do not wish to receive any newsletter. 

Please take my name off the mailing list. 

Recommendations of the Panel of Experts 
A panel of experts was formed to examine the draft 
questionnaire to ascertain face and content validity. The panel 
members were selected by the researcher. They included a county 
agent from Russell County, the county agent coordinator of the 
Barbour County Extension Service, and the district agent from the 
Tuskegee University Extension Service. This jury of experts was 
selected for the formative committee because of their educational 
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backgrounds, employment status, knowledge, and work experience in 
the extension service (see Appendix B) . 

A meeting was held with the panel to establish goals, format, 
and content of the survey questionnaire. After reviewing the 
draft questionnaire, the county agent coordinator and the district 
agent decided to maintain all closed-ended questions because they 
would be the best method for obtaining data that can be categorized 
easily (Isaac and Michael, 1995, p. 147) . Finally, all of the 
panel members agreed to reserve questions number one, two, four, 
and six for the final draft. The panel members also suggested 
keeping "checklist" items for these questions. 

For question number three, the committee suggested using a 
five point Likert scale to measure each clientele's rating for the 
newsletter instead of a four point scale. A Likert scale provides 
great flexibility since the descriptors on the scale vary to fit 
the nature of the question or statement. 

The jury decided to delete question number five from the 
questionnaire. The question was considered unnecessary. They 
believed that the question might confuse and discourage respondents 
from answering the questionnaire. 

The final draft survey contained five questions. Question one 
and two sought to answer the first goal, which was to determine if 
the clientele read the newsletter. Question three was designed to 
glean how the clientele evaluates the content and the quality of 
the newsletter. Question four was selected to learn the format and 
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readability of the newsletter. Question five was included to 
identify recipients who no longer wished to receive the Ho lp inq 
Hand newsletter. In addition to reviewing the survey draft, the 
panel provided twelve guidelines for designing the survey 
questionnaire : 

1. Use two pages to print questions to avoid cluttering. 

2. Make instructions brief and easy to understand. 

3 . Avoid abbreviated items . 

4. Use examples if the items may be difficult to understand. 

5. Keep the questionnaire as short as possible. 

6. Use a logical sequence, and group related items together. 

7. Put important items near the beginning of a long 
questionnaire . 

8. Carefully check grammar, spelling, punctuation, and other 
details . 

9. Make sure print is clear and easy to read. 

10. Number the pages and the items. 

11. Highlight words to emphasize important points. 

12. Use an attractive heading 

The panel approved the contents of the cover letter, 
reminding through telephone calls, and follow-up home visits. 

There were no changes for the cover letter (see Appendix D) . 

The panel strongly recommended field-testing the 
questionnaire and the cover letter among ten people to assure 
readability of the survey questions. The district agent of the 
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Tuskegee University Extension Service suggested including a few 
minorities in the field-test. It was also decided by the panel 
that any returned incomplete questionnaires would be considered 
invalid . 

Selection of Final Questionnaire 
The following questions were selected to evaluate the 
Helping Hand : 

1. How much of an issue of the Helping Hand do you usually 
read? 

All or almost all 

At least half 

Less than half 

Little or none 

2 . If your copy of the Helping Hand failed to arrive each 
month, would you: 

Be unable to function without it 

Be disappointed 

Notice its absence 

Never know the difference 
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3. Please rate the Helping Hand in terms of the following 
statements : 

Always Most of Sometimes Rarely Never 
the Time 

Information given 
in the Helping Hand 

is useful — — 

Information given 
in the Helping Hand 

is reliable — 

Information given 
in the Helping Hand 

is timely — 

4. Please Check your reaction to this statement: The 
newsletter is easy to understand: Unnecessary and difficult 
words are avoided. 

Strongly agree 

Agree 

Not sure 

Disagree 

5. Which of the following statements most accurately 
represents your opinion? 
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I would prefer to receive the newsletter each month. 

I would prefer to receive the newsletter once a quarter. 

I do not wish to receive the newsletter. 

Field-test of the Questionnaire 
and the Covex Letter 

Ten people were selected to field-test the cover letter and 
the questionnaire to assure readability and uniform understanding 
of survey questions. Six females and four males were selected. 
Among them, two females and one male were African-American. 

All ten people mentioned that the cover letter was easy to 
understand. Eight people completed the questionnaire in less 
than five minutes. The other two individuals took about eight 
minutes to complete it. They all were able to understand the 
questionnaire without any difficulties. 

Response to Data Collection Efforts 
Out of one hundred questionnaires mailed, thirty-five 
responses were returned within the first week. A follow-up 
telephone call was made after two weeks. Twenty responses were 
returned after the follow-up telephone call. To secure a higher 
response rate, fifteen home visits were conducted by the 
researcher to encourage respondents to return the survey 
questionnaires . Ten more responses were collected as a result of 
the home visits. One questionnaire was returned blank. One 
subject had died since the last update of the mailing list. Two 
clients declined to respond to the survey because of time 
constraints . 
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Responses to Questionnaire Items 
A total of 61 usable questionnaires were received for 
analysis. Data were analyzed using the Microsoft Works for 
Windows . 

The Amount of the Helping Hand that Readers Normally Read 



AT LEASE HALF (4.9%) 





ALMOST ALL (95.1%) 



Figure 1 . Response to Item 1. 

Ninety-five percent of all respondents indicated that they 
read all or almost all of the articles in the publication. Five 
percent indicated reading at least half. 
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The respondents were jasked to express their reaction 
if the newsletter failed^to arrive each month 
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NEVER KNOW THE DIFFERENCE (6.6%) 



NOTICE ITS ABSENCE (32.8% 




Figure 2 . Responses to Item 2. 

Sixty-one percent of the respondents indicated that they 
miss their copy of the Helping Hand if it fails to arrive each 
month. Thirty- three percent would notice its absence, and the 
rest, seven percent, would never know the difference. 
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Respondents were asked to rate the Helping Hand in terms of 
its usefulness, reliability, and timeliness of information. 
Readers 1 Rating Regarding the Usefulness 
of the Newsletter 



Figure 3.01 . Responses to item 3.01. 

Fifty-one percent indicated that they always found the 
information given in the Helping Hand useful. Sixteen percent 
found the information useful almost all the time, thirty percent 
found the information useful sometimes, and three percent found 
the information rarely useful. 



RARELY (3.3%) 



MOST OF THE TIME (16.4%) 



SOMETIMES (29.5% 




ERIC 
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Readers ' Rating Regarding the reliability 
of the Newsletter 
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SOMETIMES (23.0%) 



MOST OF THE TIME (8.2%) 



J 




ALWAYS (68.9%) 



Figure 3.02 . Responses to item 3.02. 

Sixty-nine percent of respondents indicated that the 
information given in the Helping Hand was always reliable. Eight 
percent found the information reliable most of the time, and 
twenty-three percent found the information reliable sometimes. 
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Readers ' rating Regarding the timeliness 
of the Newsletter 




Figure 3.03 . Responses to Item 3.03. 

Seventy-five percent of respondents indicated that the 
information given in the Helping Hand was timely. Seven percent 
found the information timely most of the time, and eighteen 
percent found the information timely sometimes . 
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Respondents were asked to comment on the 
format and content of the Helping Hand 



NOT SURE (1.6%) 





STRONGLY AGREE (60.7%) 



Figure 4 . Responses to Item 4. 

Sixty-one percent of the respondents strongly agreed that 
the newsletter was easy to understand and that unnecessary and 
difficult words were avoided. Thirty-eight percent agreed, and 
one percent were not sure . 
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Respondents were asked to specify how they felt 
about the frequency of the Helping Hand 
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PREFER NO NEWSLETTER (4.9%) 




MONTHLY PREFERENCE (80.3%) 



Figure 5 . Responses to Item 5. 

Eighty percent of respondents indicated that they would 
prefer to receive the newsletter each month. Fourteen percent of 
respondents preferred to receive the newsletter once a quarter, 
and five percent of respondents did not wish to receive the 
newsletter. 
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Comments 

A few respondents wrote a comment on the survey sheet, even 
though the questionnaire did not include any open-ended 
questions. 

1. "Good survey, thank you." 

2. "Keep us informed." 

3. "Thanks for sending Helping Hand in the past. I have 
enjoyed it very much, but since I have retired I do less cooking, 
etc . " 

4. "I think it has served its purposes." 

5. "Close it down as a budget cut. It has no value." 

6. "Helping Hand is an average newsle 

tter. I would make it an excellent newsletter or a dropout." 

7. "Under present budget and time constrains, I would 
eliminate it." 

8. " Due to eye problems I'm unable to read much. Thanks 
for sending your newsletter during last several years." 

9. "If my copy of the newsletter failed to arrive I will 
notice its absence, maybe not the first time but eventually I 
would wonder what happened to it." 
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Chapter 5 

DISCUSSION, CONCLUSIONS, 

IMPLICATIONS, AND RECOMMENDATIONS 
Discussion 

The purpose of this study was to evaluate the effectiveness 
of the Helping Hand newsletter. 

The objectives of this study were to ascertain readers' 
utilization of the publication and their perceptions about the 
format, appearance, and usefulness of the newsletter. 

The first research question asked: What are the valid and 
appropriate criteria to evaluate the newsletter? 

The criteria for the evaluation were established by 
reviewing the related literature (Fisher, 1990, p. 43; Shoemaker, 
1990, pp . 42-44; and by consulting with three extension 
professionals . The criteria were validated by a panel of three 
other experts . 

The second research question asked: What are the valid and 
appropriate questions to measure the readers' attitudes, the 
usefulness of the newsletter, and the suitability of the 
newsletter contents? 

The questionnaire developed for the evaluation of the 
Helping Hand was derived from literature related to various 
readership surveys. The literature review was divided into two 
broad areas. Those were: (a) readership survey and (b) 

designing survey instruments. The literature review included 
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several readership surveys conducted by the extension service, 
such as a readership survey of the Communicator by Suvedi and 
Johnson (1992) , evaluation of an Interactive Newsletter by 
Goetting and Pourroy (1991), evaluation of the Woodlander by 
Smith and Kiernan (1992), and a readership survey of the 
Messenger by Suvedi et al . (1993) . The literature review also 

included a readership survey of the alumni publication The Tower 
by Piovane (1995), evaluation of Doane ' s Agricultural Newsletter 
by Schumacher (1989), and a readership survey of Vital Signs by 
Pavlik, Vastyan, and Mahar (1990). 

A panel of experts provided guidelines to establish the 
content and format of the survey questionnaire. 

The third question asked: What is the valid and appropriate 
format for the survey? 

The panel of experts guided the selection of content and 
format for the survey questionnaire. Suggestions were also taken 
from the literature review (Paxson, 1992; Stratton and Angerosa, 
1995; Faria and Dickinson, 1992). The questionnaires were field- 
tested to assure readability and uniform understanding. 

The survey questionnaires were sent along with a current 
issue of the newsletter to a stratified random sample of 10% of 
the 1,000 recipients of the newsletter. 

Sixty-one percent of those subscribers responded to the 
questionnaire in the mail. The data were analyzed by using 
descriptive statistics. 
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The results of the newsletter evaluation provided 
unambiguous information for decisions about the newsletter's 
future. A substantial majority of the respondents indicated that 
they usually read the entire newsletter and virtually all of the 
respondents, found the newsletter easy to understand. 

Objectives of all eight procedures were successfully met. 

The newsletter was evaluated. This study was the first 
readership survey ever conducted to assess the effectiveness of 
the newsletter. This survey provided information that was not 
available before. 

The usefulness and the effectiveness of a publication 
greatly depend on how it meets and reflects the interests of its 
readers. A readership survey, therefore, is a valuable tool in 
analyzing reader opinion. Findings of the survey showed that the 
publication was needed and valued by the extension clients . The 
study also showed a satisfactory rating of the Helping Hand by 
the readers . 

A newsletter that was read and used could be an effective 
method of disseminating information; however, a newsletter that 
was not read would be too costly for the extension service to 
continue. The findings of this study would assist Barbour County 
Extension Service in reaching decisions on future directions of 
the newsletter. 

A report including conclusions and recommendations was 
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A report including conclusions and recommendations was 
submitted to the Administrator of the Tuskegee University 
Extension Service. 

Conclusions 

This study was conducted to get feedback on the Helping 
Hand , a monthly newsletter of the Barbour County Extension 
Service. The objectives of this study were to ascertain readers' 
utilization of the publication and their perceptions about the 
format and content of the newsletter. 

The study shows that a majority of the respondents read all 
or almost all of the articles in the publication. The majority 
of the readers would notice its absence if they miss an issue. 

The publication was perceived by the readers as very useful, 
timely, and reliable. The respondents read the newsletter and 
shared it with friends and neighbors. 

The Helping Hand newsletter is an effective method of 
meeting the needs of many individuals who do not have time to 
take courses or read books, yet they find time to read magazines, 
newspapers, and newsletters. 

Implications 

The conclusions of the survey have serious implications for 
decisions about the future of the Helping Hand newsletter. The 
results of the study show that a majority of the survey 
respondents prefer to receive a monthly printed newsletter. 
Informational newsletters can be an effective and cost-efficient 
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method of disseminating information. The expenditure in 
publishing the Helping Hand is justifiable mainly because the 
publication is serving its purpose, and, therefore, current costs 
are within reason. It seems that the purpose of the Helping Hand 
is being met, and that it is doing the job for which it was 
created . 

Recommendations 

Based on the findings of this study, the following 
recommendations were made: 

1. Barbour County Extension Service should continue 
publishing the Helping Hand . The Helping Hand has served as an 
important information resource to its readers, who could not 
attend extension programs. The current* cost is within reason. 

2. Conduct a follow-up survey. 
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Appendix A 

Criteria of the Survey 



49 



To establish the criteria for the survey, a literature 
review was conducted on several readership surveys . The 
researcher sought advice for establishing criteria from Mr. 
Charlie Mason, a county agent; Dr. Barbara Diffaey, an extension 
specialist; and Dr. Robert Zabawa, a Research Associate from 
Tuskegee University Extension Service. They were each provided 
with a summary of the literature review, a sample of a readership 
survey that was conducted at Michigan State University by Suvedi 
et al. (1993), and a copy of the handbook on "Research and 
Evaluation" by Issac and Michael, 1995. 

After their review and comments, the following criteria were 
established for the survey: 

1. Each Question should be related to the research goals. 

2. Questions should be relevant. 

3. Questions should be clear, concise, and unambiguous. 

4. Avoid double-barrelled questions. 

5. Respondents must be competent to answer. 

6. Avoid negative items or terms. 

7. Avoid biased items or terms. 
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Appendix B 

The Formative Committee 

A committee was formed to examine the draft questionnaire 
to ascertain face and content validity. The committee members 
were selected by the researcher. They included Mrs. Tarana Kahn, 
a county agent from Russell County Extension Service (adjoining 
County) ; Mr. James McGhee, the county agent coordinator of the 
Barbour County Extension Service; and Dr. Sharon Anderson, the 
district agent of the Tuskegee University Extension Service. 

Mrs. Tarana Kahn has been working as a county agent since 
1990. Mrs. Kahn received a master's degree in Foods and 
Nutrition from Tuskegee University. She has a monthly newsletter 
that reaches about 1500 people. She h&s published several 
publications in the area of Foods and Nutrition. 

Mr. James McGhee is the county agent coordinator of the 
Barbour County Extension Service. Mr. McGhee has been serving as 
a county agent coordinator since 1978. Mr. McGhee supervises 
four different newsletters every month that reaches approximately 
6000 people in Barbour County. He holds a master's degree in 
Agriculture . 

Dr. Sharon Anderson is the district agent of the Tuskegee 
University Extension Service. She has been appointed as a 
district agent since 1993 . She received her doctorate degree 
from the Michigan State University in Adult Education. She has 
several publications in adult education. 
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These three professionals were selected for the formative 
committee because of their educational backgrounds and work 
experience . 

The committee met once, and a design model was developed for 
the format and the contents of the questionnaire. 

The following are the format and the final questions for the 
survey that was recommended by the formative committee. 

1. Use two pages to print questions to avoid cluttering. 

2. Make instructions brief and easy to understand. 

3. Avoid abbreviated items. 

4. Use examples if the items may be difficult to 
understand . 

5. Keep the questions as short a£ possible. 

6. Use a logical sequence, and group related items 
together . 

7 . Put important items near the beginning of a long 
question . 

8. Carefully check grammar, spelling, punctuation, and 
other details. 

9. Make sure printing are clear and easy to read. 

10. Number the pages and items. 

11 . Highlight words to emphasize important points . 

12. Use an attractive heading. 
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Appendix C 

QUESTIONNAIRE 

Please respond to the following questions . Check only 
one answer for each question: 

1. How much of an issue of the Helping Hand do you 
usually read? 

All or almost all 

At least half 

Less than half 

■ Little or none 



2. If your copy of the Helping Hand failed to arrive 
each month, would you: 

Be unable to function without it 

Notice its absence 

Never know the difference 



3 . Please rate the Helping Hand in terms of the 
following statements: 



Always 

Information given in the 
Helping Hand is useful 

Information given in the 
Helping Hand is reliable 

Information given in the 
Helping Hand is timely 



Most of Sometimes Rarely Never 
the Time 
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4. Please check your reaction to this statement: The 
newsletter is easy to understand: unnecessary and 
difficult words are avoided. 

Strongly agree 

Agree 

Not sure 

Disagree 

5. Which of the following statements most accurately 
represents your opinion? 

I would prefer to receive the newsletter each month. 

I would prefer to receive the newsletter once a quarter . 

I do not wish to receive the newsletter. 



THANK YOU! 
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Appendix D 



July 13, 1995 

Barbour County Extension Office 
Post Office Box 99 
Clayton, Alabama 36016 
Telephone: (334) 775-3285 



Dear Friend: 

It has been about ten years since we started to publish the 
newsletter Helping Hand . Since then we never had a chance to 
evaluate the newsletter. In order to obtain your opinion and 
overall attitude regarding the usefulness and content of the 
Helping Hand , we are conducting a readership survey, and we would 
like to have your input . 

We need your opinion to assure thht the study represents the 
views of all readers. We hope you will choose to participate in 
this study. Your name will be conf idencial, and the information 
that you provide will be used only in combination with other 
responses . 

Thanks in advance for taking a few minutes to fill out the 
survey form. Please return it in the enclosed pre-addressed, 
stamped envelope to: Habiba N. Shaw, County Agent, Barbour County 
Extension Service, P.O. Box 99, Clayton, Alabama 36016. 

As a token of our appreciation for your help, we are 
enclosing a copy of the latest edition of the he Ipim — H&nd. 



Sincerely, 




Habiba N. Shaw 
County Agent 



ERiC 
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Appendix E 



COOPERATIVE EXTENSION PROGRAM 

TUSKEGEE UNIVERSITY 

Tuskegce University, Alabama A & M, Auburn University 
and L T . S. Department of Agriculture cooperating 

Barbour County Extension Office 
Post Office Box 99 
Clayton, Alabama 36016 

Telephone: (334) 775-3285 

" helping hand n 

HOMEMAKERS' N E W SL ETT ER BARBOUR /BULLOCK COUNTY AUGUST/SEPTEMBER, 1995 





"KEEP IT LOW-FAT” 



RAirwc ; Preheat oven to 450°F. To keep fish moist, bake it in a small amount 
of liquid — low-fat milk, seasoned water, broth or vine. Seal in moisture with 
a light bread or cracker crumb topping. 

BROILING OR GRILLING : Try a Light coating of non-fat mayonnaise or low-fat 

yogurt wi th h«»rh*- You can also baste with barbecue sauce or a mixture consist- 
ing of lemon or lime juice, herbs and a small amount of olive oil. 



POACHING: For oven poaching, place fish in a baking dish and b are ^ ^ 0 ^ 

with any combination of the following: milk, wine and water, season an 

salted water, or chicken or fish stock with vegetables. Lay a sheet of toil 
across the top of the dish and bake in preheated 400 F oven. 




STEAMHG: You can steam over plain boiling water or any of the broths listed 

for poaching. If you don*t have a fish steamer, you can devise one by *““8 
a large pan with a roasting rack that holds fish above — not in the boiling 

water. , 

*>.e Tuik«ee Unrvenuv Cooperative fcuenaon Procam often iu proeramt 10 persona mardleu 01 race, cotor. national 
inpn. sex or handicap and ii an <oual opportunity employer. ( over ) 



RP.QT rnPY AV/AII ARIF 
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MICROWAVE COOKING : It is best to check your oven's manual for seafood guide- 
lines. In general, microwave cooking times are 1/4 to 1/3 of those used for 
conventional recipes . 



Dietary fiber, often 
ables, fruits, beans, nuts, and seeds that we can't digest. There are two 
types of fiber, insoluble and soluble. Both types of fiber are important for 
a healthy diet. The National Cancer Institute reconmends between 20 to 30 grams 
of fiber a day, with an upper limit of 35 grams, from a variety of sources like 
bran cereal, whole grain breads, vegetables, and fruits. 

"DIFFERENCE BETWEEN OAT 



" FILLING UP ON FIBER" 



ailed roughage, is the part of whole grains, veget- 




BRAN AND OATMEAL" 



A whole grain is made up of the bran, the germ and the endosperm. Oat- 
meal, a whole grain, contains all these layers, while oat bran consists of 
only the bran layer of the grain. Both oatmeal and oat bran are good choices 
nutritionally. Oat bran has about two times as much soluble fiber as oatmeal. 
The vitamin and mineral content of oat bran may be slightly higher than oatmeal , 
but you can't go wrong with either choice. Below are two delicious recipes: 



/ 



Soda Bread: 




1-1/2 cups \ta le wheat flour 
1 cup all purpose flour 
1/2 cup rolled oats 
1/4 cup sugar 
1/2 teaspoon baking soda 
1-1/2 teaspoons baking powder 



1/4 teaspoon ground rinra m m 
1/3 cup raisins (optional) 
1/4 cup wahmits (optional) 
1-1/4 cups low-fat buttermilk 
1 tablespoon wgptable oil 



7 O 

Preheat oven to 375 . In large bowl, stir together flours, oats, sugar, 
baking soda, baking powder and cinnamon. If adding raisins and walnuts , mix in. 
Gradually stir in buttermilk and oil until forms ball. Knead in bowl for 30 
seconds. Spray loaf pan with cooking spray and turn dough into pan* Ba ke 40 
to 50 minutes until knife inserted in middle comes out clean. Makms*l6 slices 
and h^a 126 calories per serving. Nutrition Content : 23 g carbohydrate , 3 g 

fat, 4 g protein, 2 g dietary fiber, 78 mg sodium, 1 mg cholesterol.; 



Cantaloupe Strawberry Shake : 1 cup lide strawberries 2 Irmsponm «jgar 

1 cup cantaloupe 1/2 ccp skim milk 

Cut fruit in big chunks. Add to food processor with sugar and milk, and 
blend until frothy. Serve immediately. Serves 2. 89 calories per serving. 

Nutrition Content : 19 g carbohydrate, 1 g fat, 3 g protein, 3 g dietary fiber, 

40 mg sodium, 1 mg cholesterol. 

Sincerely, 

Habiba N. Shaw 
County Agent 

* gw 
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